The present invention relates to a method and apparatus for organizing and tracking the workflow of test question responses provided by a test taker and captured as electronic images.
The scoring of test question responses that cannot be scored by machine is typically carried out manually or by presentation on a computer monitor. Manual scoring involves a human manually scoring a physical test question response sheet. Scoring by presentation to a human of the test question responses using a computer involves scoring an electronic representation of the test question response or responses presented to a scorer via a computer monitor or other machine that can be programmed to manipulate symbols. It is the latter scoring procedure to which the present invention relates.
In order to present a test question response to a scorer viewing a computer monitor, several preparation steps typically occur to enable the scorer to receive the test question response, view it, score it and record the score with the necessary precision, speed and accuracy required in the test scoring industry. With test processing that analyzes optically imaged test question responses, scanners, facsimile machines and other optical imaging devices known to those skilled in the art are used to create an electronic image of the test question response that was provided by a test taker. The electronic images may be broken down into smaller images generally referred to as image clips. The electronic images and image clips may be stored in a computer or other storage media known to those skilled in the art. Multiple optical imaging devices of varying types are often used in a distributed test processing system. The electronic images and image clips are then often converted to data using well known and commercially available optical character recognition software, image character recognition software and other similar computer programs. The data or images, or both, can then be utilized in a number of ways to aid the test scoring process.
A problem typically faced in the test processing industry, especially when imaging of the test question responses is utilized, is the difficulty in organizing and tracking the workflow of the test question response images simply and inexpensively in a server/client workflow system. The problem is magnified when different types of optical image devices (e.g., different types of scanners) are used to generate the electronic images. It will be appreciated that accuracy and reliability are of primary concern in the test question response industry. It is critical that all test items received are processed. Often, a client will want the test answer documents analyzed in different sub-sets. The result to-date often was a decrease in volume and turn-around time of customer projects.
The present test question response workflow tracking method and system increases the volume and speed of test question responses processed by improving the distribution and tracking of the electronic images of the test question responses. The tracking method and system also ensure that all test items received are processed.
In the method, electronic images of test question responses are stored on a server subsystem. Depending on what kind of optical imaging equipment is used, each image covers either one page (one side of a sheet of paper) or a small region of a page known as a clip. A clip contains one item of information, such as the answer to one test question or some information about the person taking the test.
The test question responses are organized in batches for production of the electronic images, and the association of each image with its batch is preserved during all subsequent processing. Images are held in TIFF (tagged image file format) files and each image is tagged with an individual digital identifier (IDI) that uniquely identifies it. The IDI contains all the information about the image that the system requires for displaying it in the correct context. Holding this information in the IDI, rather than in a database, permits faster retrieval of the data when the image needs to be displayed. It has the added benefit of avoiding any risk of the image becoming separated from the relevant data.
Human operators using computers connected to the server through a network may view the images and may enter data that they derive from the images into a corresponding database record. Information within the IDI is used to link the image to the correct database record.
When the processing of a batch of test question responses is complete, a report may be generated from the database. The completeness of each batch is ensured by comparing the number of test question responses processed with the number originally assigned to the batch.